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(57) Abstract: A video communication system that uses multiple streams to carry digital video. The use of multiple streams ensures 
that errors in one or more of the multiple streams do not prevent reconstruction of remaining ones of the multiple streams. This 
enables an error free display of the digital video at a reduced frame rate during the loss of a subset of the streams. In addition, the 
multiple streams provide past and future frames that may be used to recover lost frames and thereby recover the lost stream. 
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VIDEO COMMUNICATION USING MULTIPLE STREAMS 

Field of Invention 

The present invention pertains to the field of 
5- video communication.. More particularly, this 
invention relates to video communication using 
multiple encoded streams. 

Art Background 

10 A wide variety of video communication systems 

including digital television systems, Internet video 
systems, and wireless video systems commonly employ 
communication links to carry digital video. 
Typically in such systems, a digital video originates 

15 in a sender and is transferred to one or more 

receivers via one or more communication links. In a 
digital television system, for example, a broadcast 
station as a sender usually transmits digital video 
to one or more digital television receivers via over- 

20 the-air or cable communication links. In an Internet 
video system, as another example, a web server 
typically acts as a sender that transfers digital 
video to one or more web client receivers via the 
Internet. 

25 

Digital video in such systems is typically 
arranged as a series of video frames. The video 
frames usually occur at high enough frame rate to 
enable a viewer to perceive full motion video when 
30 the video frames are rendered on a display. For 

example, a frame rate between 15 and 30 frames per 
second is common in video communication systems. 
Each video frame usually carries an array of pixel 
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data. The relatively high amount of pixel data 
contained in each video frame combined with a frame 
rate high enough to provide full motion video would 
typically impose a relatively high bandwidth 
5 consumption on a communication link. 

Prior video communication systems commonly 
employ video compression to reduce the bandwidth 
consumption of digital video. Typically, a sender 

10 includes an encoder that generates a series of 

encoded frames in response to a series of original 
video frames. Each receiver usually includes a 
decoder which reconstructs the original series of 
video frames from the encoded frames. The total 

15 amount of data carried in the encoded frames is 

usually significantly less than the total amount of 
data in the corresponding original video frames. 

The encoded frames in prior video compression 
20 methods typically include frames which carry all of 
the information needed to reconstruct the 
corresponding original video frame. Such frames may 
be referred to as intra frames (I-frames) . In 
addition, the encoded frames in prior video 
25 compression methods typically include frames for 

which reconstruction of the corresponding original 
video frame depends on a prior encoded frame from the 
series of encoded frames. Such frames may be 
referred to as predicted frames (P-frames) because 
30 they are commonly generated by an encoder using a 
prediction loop. 
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Typically, the amount of data carried in an I- 
frame is significantly greater than the amount of 
data carried in a P-frame. Therefore, bandwidth 
savings is usually enhanced if a greater percentage 
5 of the encoded frames are P-frames. Unfortunately, 
when using prediction the loss of a P-frame or I- 
frame during transmission typically prevents the 
reconstruction of the corresponding original video 
frame as well as the reconstruction of a sequence of 

10 subsequent P-frames before a next I-frame. The loss 
of a sequence of P-frames usually has negative 
effects on the reconstructed digital video. Such 
effects may result in freeze frame or the appearance 
of displayed artifacts- These negative effects are 

15 typically worsened in systems which use a relatively 
large number of P-frames between I-frames due to 
communication bandwidth constraints. 
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SUMMARY OF THE INVENTION 

A video communication system is disclosed that 
uses multiple streams to carry digital video. The 
5 use of multiple streams ensures that errors in one or 
more of the multiple streams do not prevent 
reconstruction of remaining ones of the multiple 
streams. This enables an error free display of the 
digital video at a reduced frame rate during the loss 
10 of a subset of the streams. In addition, the 

multiple streams provide past and future frames that 
may be used to recover lost frames. 

Other features and advantages of the present 
15 invention will be apparent from the detailed 
description that follows. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is described with respect 
to particular exemplary embodiments thereof and 
reference is -accordingly made to the drawings in 
which : 

Figure 1 illustrates a video communication 
system according to the present teachings; 

Figure 2 illustrates the advantages yielded by 
the separate streams when an error occurs in 
transmission of one of the streams; 

Figure 3 illustrates an embodiment of a receiver 
which includes a frame recovery; 

Figure 4 illustrates an example in which past 

and future frames are selected for use in recovering 

a lost frames- 
Figure 5 shows another embodiment of a video 

communication system according to the present 

teachings. 



DETAILED DESCRIPTION 



Figure 1 illustrates a video communication 
system 10 according to the present teachings. The 
video communication system 10 includes a sender 30 
that communicates a series of original video frames 
40 to a receiver 32 using a set of streams 42. In 
this embodiment, the streams 42 include a pair of 
streams 54 and 56 which carry encoded odd and even 
frames, respectively, from the original video frames 
40. 

In this embodiment, the sender 30 includes a 
frame separate block 12 that separates the original 
video frames 40 into a series of odd video frames 50 
and a series of even video frames 52. For example, 
if the original video frames 40 are a series of 
frarne^ frame 2/ frame 3 , frame 4/ frame 5 , frame 6/ etc., 
then the odd video frames 50 are a series of frame w 
frame 3 , frame 5 , etc. and the even video frames 52 are 
a series of frame 2 , frame 4 , frame 6 , etc. In some 
embodiments, the frame separate block 12 performs 
pre-processing such as temporal low-pass filtering or 
noise reduction before extracting the frames for each 
stream. 

The sender 30 includes an encode block 14 that 
generates the encoded odd frames 54 in response to 
the odd video frames 50. The encoded odd frames 54 
include I-frames and P-frames. In response to the 
example series frame lf frame 3 , frame 5 , etc., the 
encode block 14 may generate an I-frame I lr followed 
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by P-frames P 3 , P 5 ", etc. P 3 depends on I x and P 5 
depends on P 3 , etc. 



Similarly, the sender 30 includes an encode 
5 block 16 that- generates the encoded- even f rames 56 in 
response to the even video frames 52. In response to 
the example series frame 2 , frame 4 , frame 6/ etc., the 
encode block 14 may generate an I-frame I 2 , followed 
by P-frames P 4/ P 6 , etc. P 4 depends on I 2 and P 6 
10 depends on P 4f etc. 

Alternatively, in response to the example series 
frame w frame 3 , frame 5 , etc. the encode block 14 may 
generate I lf P 3 , P 5 , etc, and in response to the 
15 example series frame 2 , frame 4/ frame 6 , etc. the encode 
block 16 may generate P 2 , P 4 , P 6 , etc, where P 3 depends 
on I x and P 5 depends on P 3 , etc., and where P 2 depends 
on I x and P 4 depends on P 2 and P 6 depends on P 4/ etc. 

20 In another alternative, the I-frames in the 

encoded odd and even frames 54 and 56 may be offset 
with respect to one another. For example, the encode 
block 14 may generate I lf P 3 , P 5 , etc, and the encode 
block 14 may generate P 2 , P 4 , P 6 , I 8/ P i0 / P12/ etc, 

25 where P 3 depends on I x and P 5 depends on P 3 , etc. and 
where P 2 depends on I r and P 4 depends on P 2 and P 6 
depends on P 4 and P 10 depends on I 8 etc. In such an 
alternative, it may be preferable that the I-frames 
in the encoded even frames 56 be evenly spaced 

30 between the I-frames in the encoded odd frames 54 and 
visa versa. 
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The communication block 18 carries the streams 
of encoded odd and even frames 54 and 56 to the 
receiver 32. It is preferable that the streams of 
encoded odd and even frames 54 and 56 be carried in 
5 separate channels or communication paths so that 

disruptions that cause errors in the transmission of 
one of the streams do not hinder the transmission of 
the other. For example, if the communication block 
18 is implemented as a packetized communication 

10 network then the stream of encoded odd frames 54 

should be sent in packets that are separate from the 
packets that carry the stream of encoded even frames 
56. As another example, if the communication block 
18 is implemented as a radio frequency network then 

15 the stream of encoded odd frames 54 should be 

transmitted in a frequency band that is different 
from the frequency band that carries the stream of 
encoded even frames 56. 

20 The receiver 32 includes a decode block 20 that 

generates a series of reconstructed even video frames 
60 in response to the encoded even frames 56. The 
receiver 32 also includes a decode block 22 that 
generates a series of reconstructed odd video frames 

25 62 in response to the encoded odd frames 54. The 

decoding method implemented in the decode block 20 is 
adapted to the encoding method in the encode block 
16. Similarly, the decoding method implemented in 
the decode block 22 is adapted to the encoding method 

30 in the encode block 14. 

For example, if the encode block 16 generates P- 
frames in the encoded even frames 56 that carry 
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motion estimation and error information according to 
a prediction loop, then the decode block 20 uses the 
motion estimation and error information from the 
received P-frames in its own prediction loop 'to 
5 generate the reconstructed even video frames 60. The 
motion estimation parameters and error information 
parameters may be defined in a video compression 
standard. Examples of these standards are numerous 
and include the various motion picture engineering 
10 group (MPEG) standards such as MPEG-1, MPEG-2, MPEG- 
4, as well as H.261, and H.263. 

In such embodiments, the prediction loops in the 
encode blocks 14-16 and the decode blocks 20-22 

15 continually update their states with each encoded 
frame generated or received. For example, the 
prediction loop in the encode block 14 updates its 
state for each encoded odd frame 54 it generates and 
the decode block 22 updates its state after decoding 

20 the odd frames 54. Each prediction loop contains 
information which may be referred to as its state. 
The state of a prediction loop is used to form the 
prediction of the current frame to be encoded. One 
component of the state is the previous coded frame. 

25 The state varies with each encoded frame. 

Each encode block 14-16 has a prediction loop 
with its own state. Each encode block 14-16 forms a 
prediction based on its state and codes and transmits 
30 the error signal along with side information such as 
motion vectors and inter/intra decisions for each 
block. This information describes how to form a 
prediction from the state. Each decode block 20-22 
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forms a prediction based on its state and the side 
information and then adds the received decoded error 
signal to reconstruct the current frame. The 
encode/decode block pairs 14/22 and 16/20 should have 
5 the same state. Otherwise their predictions will 
differ and they will lose tracking and may produce 
significant visual degradations in the reconstructed 
video frames 44 . 

10 Prior video compression methods usually have a 

single prediction loop at the sender and the receiver 
with a single state or other information which if 
lost results in different predictions in the sender 
and receiver, thereby often leading to significant 

15 visual degradations. The present techniques use 
multiple prediction loops each with its own state 
information. Therefore, the current teachings may be 
characterized as video communication using multiple 
state streams and the streams 54-56 may each be 

20 referred to as a state stream. 

The receiver 32 includes a merge block 24 that 
combines the reconstructed even and odd video frames 
60-62 into a series of reconstructed video frames 44 

25 in the proper frame order. For example, the merge 

block 24 combines the reconstructed odd video frames 
62 that include the reconstructed full frames frame lf 
frame 3 , frame 5 , etc., with the reconstructed even 
video frames 60 that include the reconstructed full 

30 frames frame 2 , frame 4/ frame 6/ etc. to provide the 

reconstructed video frames 4 4 which include the full 
frames frame lf frame 2 , frame 3 , frame 4 , frame 5 , frame 6/ 
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etc. The reconstructed video frames 4 4 may be 
provided to a real-time display (not shown) . 

The merge block 24 may also perform post- 
processing such as known processing- for artifact - 
reduction or temporal filtering across the streams 
54-56 to reduce distortions in each stream 54-56 
and/or produce uniform quality across the streams or 
temporal frame-rate conversion. 

Figure 2 illustrates the advantages yielded by 
the separate streams 54 and 56 when an error occurs 
during transmission between the sender 30 and the 
receiver 32. In this example, P 3 depends on I x and P 5 
depends on P 3/ etc., and P 2 depends on I i and P 4 
depends on P 2 and P 6 depends on P 4 , etc. 

At time t w an error that occurs in the 
transmission of the encoded odd frames 54 prevents 
the decode block 22 from properly reconstructing P s . 
In terms of a prediction loop, the decode block 22 
cannot accurately decode frame P 5 and therefore has 
incorrect state as compared to the encode block 14. 
As a consequence, the decode block 22 cannot 
reconstruct P 5 through P l3 which are successively 
dependent on one another. The state for the decode 
block 22 is not reinitialized, in this example, until 
the I-frame I 15 is received at time t 2 . 

Between times t x and t 2/ the decode block 20 
continues to update its state with information 
carried by P 4 , P 6 , P 8 , etc. and is unaffected by the 
loss of P 5 . As a consequence, the reconstructed video 
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frames 4 4 carry the reconstructed even video frames 
60 between times t x and t 2 . This provides a display 
at one half the normal frame rate of the original 
video frames 40. This provides a better video image 
5 than prior systems which would freeze the video or 

which would perform error concealment between times t x 
and t 2 . 

In an embodiment having three separate streams, 
10 a loss of an I-frame or a P-frame in one of the 

streams yields a frame rate of two-thirds the normal 
rate. An embodiment having four separate streams 
yields a frame rate of three-fourths the normal rate 
upon a loss of a P-frame in one of the streams, etc. 

15 

Figure 3 illustrates an embodiment of the 
receiver 32 which includes a frame recovery block 26, 
The frame recovery block 26 recovers the lost P-frame 
P 5 of the stream 54 using the information provided by 

20 the stream 56. The recovery of P 5 enables the decode 
block 22 to continue with its prediction loop by 
correctly updating its state information and 
reconstruct P 7 through P 13 . The fact that the 
reconstruction of the stream 56 is unaffected by the 

25 loss of P 5 enables the frame recovery block 2 6 to use 
past and future frames with respect to the lost frame 
P 5 to recover frame P 5 . The past frames P 2 and P 4 and 
the future frames P 6 through P 14 are available to the 
frame recovery block 26 as well as P x and P 3 from the 

30 odd stream 54. Any accurately decoded frame from any 
of the streams 54-56 may be used by the frame 
recovery block 26. For example, all previous even 
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and odd frames in the streams 54 and 56 as well as 
future even frames in the stream 56. 

For example, the frame recovery block 26 may 
5 recover the full frame version of P 5 by averaging 
frame 4 and frame 6 which are the reconstructed full 
frame versions of P 4 and P 6 . The reconstructed full 
frames frame 4 and frame 6 are generated by the decode 
block 20 as part of the reconstructed even video 
10 frames 60. The frame recovery block 26 may average 
any combination of the reconstructed versions of P 2 , 
P 4 , and P 6 through P 14 as well as ? x and P 3 . Such 
averaging may be appropriate when the video image 
carried in the streams 42 is static or slowly moving. 

15 

Alternatively, the frame recovery block 26 may 
recover the full frame version of P 5 by interpolating 
or estimating the full frame version of P 5 using any 
combination of the reconstructed full frame versions 

20 of P 2 , P 4 , and P 6 through P 14 as well as P x and P 3 . 

Numerous motion estimation methods may be used to 
estimate the motion between the frames including 
block-based methods, constraint equation methods, 
pel-recursive methods, phase-correlation methods, 

25 Bayesian methods, and object-based methods. The full 
frame version of P 5 may be estimated by performing 
motion-compensated interpolation using motion 
estimates and by applying appropriate linear or 
nonlinear filtering along the motion trajectories. 

30 The interpolation may also account for covered and 
uncovered areas within the frame by appropriately 
choosing to use only future or previous frames to 
estimate the appropriate areas. 
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In another alternative, the frame recovery block 
26 may recover the full frame version of P s by 
interpolating or estimating the full frame version of 
P 5 using the motion vectors contained in any 
5 combination of the P-frames P 2 , P 4 , and P 6 through P 14 
as well as P 1 and P 3 . In addition, other coded 
information from these past and future I-frames and 
P-frames may be used. The use of coded information 
in the bit-stream may help reduce the complexity of 
10 state recovery in the receiver 32. 



The streams 54-56 my be coded in a coordinated 
manner so that if one is lost the coded information 
in the other is useful for recovering the lost 
15 sequence. For example, motion vectors for the stream 
54 may be computed while accounting for the stream 56 
so that if the stream 56 is lost the motion vectors 
from the stream 54 may be used to enable accurate 
recovery. 

20 

State recovery may be performed using any subset 
of correctly decoded frames. The lost frame may be 
estimated by replacing it by another correctly 
decoded frame, by replacing it by a motion- 
25 compensated version of a correctly decoded frame, or 
by replacing it via motion-compensated interpolation. 

The encode blocks 14-16 may send side 
information to the decode blocks 20-22 for each frame 
30 in the streams 54-56 that informs the frame recovery 
block 26 of how to perform state recovery if the 
corresponding frame is lost. 
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Known error concealment methods may be applied 
using both previous and future frames to produce 
improved concealment of the lost frame. 

In an embodiment in which the streams 54-56 
include bidirectionally predicted frames (B-frames), 
the frame recovery block 26 may optionally recover 
and display lost B-frames, This is optional since no 
other frames depend on a lost B-frame. 

Figure 4 illustrates an example in which the 
frame recovery block 26 selects past and future 
frames to be used in recovering a lost frame while 
taking into account scene changes. In this example, 
a communication error at time t 10 causes the loss of 
the encoded version of frame 5 from the encoded odd 
frames 54. Since frame3 is a P-frame and frame 4 is 
an I-frame, it is concluded that a scene change may 
have occurred between frame3 and frame4 and that 
therefore the lost frames was coded by the encode 
block 14 as an I-frame to accommodate the scene 
change. The frame recovery block 26 selects any 
combination of the reconstructed versions of I 4/ P 6/ 
P 8 , etc. to be used in the recovery of frame 5 but not 
P 2 or P 3 because they were probably associated with a 
previous scene. 

This method may also be used in instances where 
there is a P-frame for which most of its blocks are 
coded as I-blocks as opposed to P-blocks. For 
example, if P 3 corresponds to a typical P-frame where 
most of its blocks are coded as P-blocks and frame 4 is 
also a P-frame but it has a significant percentage of 
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its blocks codes as I-blocks, then this indicates 
that there is significant new imagery in the I-block 
parts of frame 4 and that that information, rather than 
corresponding areas of P 3 should be used for 
5 recovering frame 5 . A P-frame may have I-blocks when 
those blocks cannot be accurately predicted from the 
previous frame, for example when there is significant 
motion in a part of frame or there is new imagery in 
part of the frame. 

10 

In some embodiments, a communication channel 
exists that enables the receiver 32 to transmit 
information back to the sender 30. Such a 
communication channel is often called a back channel. 

15 In such embodiments, the frame recovery block 26 uses 
the back channel to notify the sender 30 that an 
error caused the loss of frame 5 . The back channel may 
be used by the receiver 32 to inform the sender 30 as 
to the method used in frame recovery. While the 

20 frame recovery block 26 recovers frame 5 using selected 
past and future frames as described above, the sender 
30 uses those same past and future frames and 
recovery method to determine the recovered frame 5 . 
The sender 30 then compares the recovered frame 5 to 

25 the actual frame 5 that was previously encoded and 
transmitted to determine the quality of recovery 
performed by the frame recovery block 26. If the 
quality of recovery is not sufficient then the sender 
30 encodes and transmits an I-frame in the encoded 

30 odd frames 54 to reinitialize the state of the decode 
block 22. In an alternative to sending an entire I- 
frame, the sender 30 may determine which sub areas of 
the recovered frame 5 are poor and send a frame that is 
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intra for those areas and predictive for the 
remaining areas. The encoder may choose to replace 
its state with the recovered frame and then continue 
to code and form predictions in a known manner. 

5 - 

The blocks 12-26 each represent hardware and/or 
software elements for performing their particular 
functions. For example, the frame separate block 12 
may be implemented with a hardware multiplexor 

10 circuit or may be implemented with equivalent 

functionality in software/firmware for a processor in 
the sender 30. The merge block 24 may be implemented 
with a hardware demultiplexer or equivalent 
software/firmware for a processor in the receiver 32. 

15 The encode blocks 14 and 16 may be implemented with 

separate hardware encoders, or with software/firmware 
executed by a processor in the sender 30, or with a 
combination thereof. The decode blocks 20-22 my be 
similarly implemented. The functionality of the 

20 frame recovery block 26 may be implemented in the 

hardware and/or software of the decode blocks 20-22 
or in separate hardware and/or software/firmware. 

The implementation of the communication block 18 
25 includes hardware and/or software/firmware elements 
that are adapted to the type of communication 
employed in the video communication system 10. For 
example, if the video communication system 10 uses 
Internet communication, then the communication block 
30 18 includes the hardware elements in the sender 30 
and the receiver 32 for Internet access as well as 
software elements for implementing Internet 
communication protocols in the sender 30 and the 
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receiver 32. In another example, if the video 
communication system 10 uses wireless communication, 
then the communication block 18 includes the hardware 
elements in the sender 30 and the receiver 32 for 
5 wireless transmission as well as software/firmware 
elements for implementing wireless communication 
protocols such as cellular in the sender 30 and the 
receiver 32. 

10 Figure 5 shows another embodiment of the video 

communication system 10 in which the sender 30 uses a 
single encoder 70 and in which the receiver 32 uses a 
single decoder 72 to handle the multiple streams 42. 
In response to the original video frames 40, the 

15 encoder 70 generates a series of encoded frames 41 
that carry encoded versions of the original video 
frames 40. A frame separate block 74 in the sender 
30 separates the encoded frames 41 into the multiple 
streams 4 2 which are the encoded odd and even frames 

20 54 and 56. A merge block 76 in the receiver 32 

reassembles in proper order the series of encoded 
frames 41 from the multiple streams 42. The decoder 
72 generates the reconstructed video frames 44 and 
may perform frame recovery as previously described. 

25 

This embodiment saves hardware in the sender 30 
and the receiver 32 in comparison to an embodiment in 
which separate encoder/decoder pairs are used for 
each of the streams 42. This embodiment is suitable 
30 for encoding and decoding according to compression 
standards in which any given encoded frame in the 
series of encoded frames 41 can depend on any other 
preceding frame in the series of encoded frames 41. 
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This allows P 8 to depend on P 6 which depends on P 4 , 
etc. 

Other embodiments may employ more that two 
5 . streams. For example, a set of three streams may 

carry every encoded third frame of the original video 
frames 4 0 or a set of four streams may carry every 
fourth encoded frame of the original video frames 40, 
etc. 



10 



15 



20 



25 



Still other embodiments may employ streams made 
up of subsets of frames which are not uniform. For 
example, the stream 54 may contain frame lf frame 2 , 
frame,, frame 5 , frame,, frame 8 , and the stream 56 may 
contain frame 3 , frame 6 , frame 9 , etc. The streams 54- 
56 may also contain fields. For example, one of the 
streams 54-56 may contain even fields and the other 
of the streams 54-56 may contain odd fields. Some 
frames may also be coded into more than one stream. 

In still other embodiments, the separation of 
frames among the streams 54-56 may be determined 
dynamically based on the content of the original 
video frames 40. ...... 



The present techniques may be applied to subsets 
of the original video frames 40. For example, each 
of the original video frames 40 may be decomposed 
into four quadrants and each quadrant may be coded 
30 using two or more separate streams. 

In addition, the number of multiple streams may 
be chosen adaptively and may be changed dynamically 
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based on the communication conditions. For example, 
when the communication is very good and there are 
very few losses it may be appropriate to have two 
streams or only a single stream. If the 
5 communication becomes worse with many losses it may 
be appropriate to change the number of streams to 
three or four. The receiver 32 may notify the sender 
30 of communication losses using, for example, a back 
channel. If communication losses are relatively low 

10 then the frame separate block 12 may send the 

original video frames 40 to one of the encode blocks 
14-16. If communication losses are relatively high 
then the frame separate block 12 divides the original 
video frames among the encode blocks 14-16. Similar 

15 functionality may be implemented in the frame 

separate block 74 to adapt to changing communication 
conditions . 

The foregoing detailed description of the 
20 present invention is provided for the purposes of 

illustration and is not intended to be exhaustive or 
to limit the invention to the precise embodiment 
disclosed. Accordingly, the scope of the present 
invention is defined by the appended claims. 
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CLAIMS 

What is claimed is: 

5 i. a video communication system, comprising: 

sender having means for generating multiple 
streams in response to a stream of video frames; 

receiver having means for reconstructing the 
stream of video frames in response to the multiple 
10 streams such that errors in one or more of the 

multiple streams do not prevent reconstruction of 
remaining ones of the multiple streams. 

2. The video communication system of claim 1, 
15 wherein the means for generating multiple streams 

comprises : 

a set of encoders each corresponding to one of 
the multiple streams; 

means for separating the stream of video frames 
20 into a stream of video frames for each encoder. 

3. The video communication system of claim 2, 
wherein the means for reconstructing the stream of 
video frames comprises : 

25 a set of decoders each for generating a stream 

of reconstructed video frames in response to one of 
the multiple streams; 

means for merging the streams of reconstructed 

video frames. 

30 

4. The video communication system of claim 1, 
wherein the means for generating multiple streams 
comprises : 
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encoder that generates a stream of encoded 
frames in response to the stream of video frames; 

means for separating the stream of encoded 
frames into the multiple streams. 

5 

5. The video communication system of claim 4, 
wherein the means for reconstructing the stream of 
video frames comprises : 

means for merging the multiple streams of 
10 encoded video frames. 

decoder for generating a stream of reconstructed 
video frames in response to the merged streams. 

6. The video communication system of claim 1, 
15 wherein the receiver further comprises means for 

recovering a frame lost from one of the multiple 
streams using one or more decoded frames from 
remaining ones of the multiple streams. 

20 7. The video communication system of claim 1, 
wherein the receiver further comprises means for 
estimating a frame lost from one of the multiple 
streams using one or more decoded frames from 
remaining ones of the multiple streams. 

25 

8. The video communication system of claim 1, 
wherein the means for generating multiple streams 
comprises means for decomposing each video frame into 
a set of quadrants and means for generating multiple 

30 streams for each quadrant. 

9. The video communication system of claim 1, 
wherein the means for generating multiple streams 
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comprises means for adapting the number of the 
multiple streams to communication conditions between 
the sender and the receiver. 

5 1-0. The video communication system of claim 1, 
wherein the sender further comprises means for 
recovering a frame lost from one of the multiple 
streams using one or more decoded frames from 
remaining ones of the multiple streams. 



10 



15 



20 



25 



30 



11. The video communication system of claim 10, 
wherein the sender further comprises means for 
encoding the multiple streams in response to the 
recovered lost frame . 

12. A method for video communication, comprising the 
steps of: 

generating multiple streams in response to a 
stream of video frames; 

reconstructing the stream of video frames in 
response to the multiple streams such that errors in 
one or more of the multiple streams do not prevent 
reconstruction of remaining ones of the multiple 
streams . 

13. The method of claim 12, wherein the step of 
generating multiple streams comprises the steps of: 

separating the stream of video frames into a 
stream of video frames for each stream; 

encoding each stream separately. 



BNSDOCID: <WO 0122735A1_L> 



WO 01/22735 PCT/US00/26294 

- 24 - 

14. The method of claim 13, wherein the step of 
reconstructing the stream of video frames comprises 
the steps of: 

generating a stream of reconstructed video 
5 frames in response to one of the multiple streams; 

merging the streams of reconstructed video 
frames . 



15. The method of claim 12, wherein the step of 

10 generating multiple streams comprises the steps of: 
generating a stream of encoded frames in 
response to the stream of video frames; 

separating the stream of encoded frames into the 
multiple streams. 

15 

16. The method of claim 15, wherein the step of 
reconstructing the stream of video frames comprises 
the steps of: 

merging the multiple streams of encoded video 
20 frames. 

generating a stream of reconstructed video 
frames in response to the merged streams. 

17. The method of claim 12, further comprising the 
25 step of recovering a frame lost from one of the 

multiple streams using one or more decoded frames 
from remaining ones of the multiple streams. 

18. The method of claim 12, further comprising the 
30 step of estimating a frame lost from one of the 

multiple streams using one or more decoded frames 
from remaining ones of the multiple streams. 
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19. The method of claim 12, wherein the step of 
generating multiple streams comprises the steps of 
decomposing each video frame into a set of quadrants 
and generating multiple streams for each quadrant. 

5 

20. The method of claim 12, wherein the step of 
generating multiple streams comprises the step of 
adapting the number of the multiple streams to 
communication conditions between a sender and a 

0 receiver. 

21. The method of claim 12, wherein the step of 
generating multiple streams comprises the steps of 
determining a recovered frame lost from one of the 

.5 multiple streams using one or more decoded frames 
from remaining ones of the multiple streams and 
encoding the multiple streams in response to the 
recovered lost frame. 
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